Apparatus and method for detecting falling object

ABSTRACT

Disclosed are an apparatus and a method for detecting a fallen object which adjust a passenger&#39;s seat to easily pick up the fallen object in the vehicle. A fallen object detecting apparatus according to one embodiment of the present disclosure is an apparatus for detecting a fallen object in a vehicle which includes a camera configured to generate at least one image of an inside of the vehicle; an image identifier configured to identify at least one passenger and at least one object from the image, and to determine a location of the fallen object in response to a falling of the object in the vehicle; and a controller configured to provide the determined location of the fallen object via at least one component located in the vehicle and adjust a seat of the passenger based on the location of the fallen object and a condition of the passenger.

CROSS-REFERENCE TO RELATED APPLICATION

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit ofearlier filing date and right of priority to Korean Patent ApplicationNo. 10-2019-0101906, filed on Aug. 20, 2019, the contents of which arehereby incorporated by reference herein in its entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to an apparatus and a method fordetecting a fallen object which adjust a passenger's seat to easily pickup the object when the object falls in the vehicle.

2. Description of the Related Art

In accordance with the development of communication technologies,cellular phones may provide various services (for example, voice call orvideo call) anytime, anywhere without being limited to time and places.Therefore, a driver may use a cellular phone even during the driving.

However, when the driver cannot immediately recognize a location of thedropped cellular phone after dropping the used cellular phone whiledriving, the driver moves to search for the cellular phone and thuscannot concentrate on the driving, which may cause a dangeroussituation.

Therefore, for the purpose of convenience and safety of the driving, alot of cradles which fix the cellular phones in the vehicle so as not todrop the cellular phones in the vehicle have been launched.

Further, a related art suggests a cellular phone cradle for a vehicle inwhich a support which vertically operates is installed at an upper edgeof the cradle to support an upper portion of the cellular phone and astand which horizontally changes the position is installed at both sidesof a lower edge of the cradle to hold the cellular phone regardless thesize of the cellular phone.

However, when the driving speed is high or the vehicle travels on anon-paved road, the cellular phone may be easily separated from thecradle to fall in accordance with the increase of the movement of thevehicle.

Therefore, a technique which provides the location of the cellular phonewhen the cellular phone falls in the vehicle to allow the user to easilypick up the cellular phone and concentrate on the driving is necessary.

RELATED ART DOCUMENT Patent Document

Patent Document: Korean Unexamined Utility Model Publication No.20-2012-0005735

SUMMARY OF THE INVENTION

An object of one embodiment of the present disclosure is to provide anapparatus and a method for detecting a fallen object which when it isdetermined that an object falls in the vehicle, provide a location ofthe fallen object through an in-vehicle component (for example, aspeaker or an AVN screen) or an internal display and adjust a seat ofthe passenger in the vehicle based on the location of the fallen objectand a physical condition (for example, arm length, leg length, or handlength of the passenger) of the passenger to easily pick up the fallenobject.

An object of one embodiment of the present disclosure is to prevent thefalling of the object in advance by suggesting to move the object to asafe area where it is determined that the object will not fall when theobject is predicted to fall in the vehicle, thereby preventing damage tothe object.

An object of one embodiment of the present disclosure is to provideguidance of a time to pick up the fallen object by notifying to pick upthe object which falls in the vehicle in a stable driving environment ornotifying to pick up the object which falls in the vehicle, in advance,when an unstable driving environment is expected.

Further, an object of one embodiment of the present disclosure is toprovide notification information about the fallen object through anin-vehicle component when the passenger did not pick up the fallenobject even after completing the driving of the vehicle to prevent thefallen object from being left in the vehicle.

According to an aspect of the present disclosure an apparatus fordetecting a fallen object in a vehicle includes a camera configured togenerate at least one image of an inside of the vehicle; an imageidentifier configured to identify at least one passenger and at leastone object from the image, and to determine a location of the fallenobject in response to a falling of the object in the vehicle; and acontroller configured to provide the determined location of the fallenobject via at least one component located in the vehicle and adjust aseat of the passenger based on the location of the fallen object and acondition of the passenger.

In one embodiment of the present disclosure, the controller isconfigured to set a range in which the passenger is capable of pickingup the object based on the physical condition of the passenger includingat least one of arm length, leg length, or hand length, and in responseto the location of the object being out of the set range, adjust atleast one of position, height, or angle of the seat of the passenger.

In one embodiment of the present disclosure, the image identifier isconfigured to track a movement of the object from the image generatedperiodically by the camera and determine, as a tracking result, that theobject has fallen based on the object dropping by a predetermineddistance or more or dropping to a floor of the vehicle.

In one embodiment of the present disclosure, the controller isconfigured to restrict an adjusting range of the seat of the passengerwithin a range in which the object does not become damaged, based on thelocation of the fallen object.

In one embodiment of the present disclosure, the image identifier isconfigured to track the movement of the object from the image generatedperiodically by the camera, and in response to the tracked object beingpredicted to fall based on at least one of movement or shape of thetracked object, the controller is configured to suggest moving theobject to a safe area, where it is determined that the tracked objectwill not fall, before the tracked object falls.

In one embodiment of the present disclosure, the controller isconfigured to suggest picking up the fallen object based on the vehiclestopping or being in a sector where the vehicle drives at apredetermined speed or lower, or configured to suggest immediatelypicking up the fallen object based on the vehicle being expected to turnsharply or drive on an irregular road within a predetermined time orbased on the vehicle being expected to rapidly decelerate or accelerate,based on navigation information of the vehicle.

In one embodiment of the present disclosure, based on the fallen objectbeing a communication device capable of being connected to the componentin the vehicle via short range wireless communication, the controller isconfigured to determine whether the object is connected to the componentin the vehicle. As a determination result, in response to the objectbeing connected to the component, the controller is configured totemporarily stop providing services of the fallen object and thecomponent. In response to the object being picked up by the passenger,the controller is configured to control the object and the component tocontinuously provide the service.

In one embodiment of the present disclosure, the image identifier isconfigured to identify whether the passenger picks up the fallen objectfrom the image periodically generated by the camera. The controller isconfigured to restrict an adjusting of the seat and suggest picking upthe fallen object in response to a command for adjusting the seat to arange in which the fallen object becomes damaged being input in a statewhere it is identified that the fallen object is not picked up by thepassenger.

In one embodiment of the present disclosure, in response to the locationof the fallen object being out of the range in which the passenger iscapable of picking up the object, the controller is configured to movethe fallen object to a pick up range by using a tool installed in thevehicle or change a driving direction or speed of the vehicle to movethe fallen object to the pick up range.

In one embodiment of the present disclosure, the image identifier isconfigured to determine, from the image of the fallen object, whether itis necessary to pick up the fallen object by applying a deep neuralnetwork model trained for determining whether the object is necessaryfor the passenger. The controller is configured to restrict theproviding of the location of the object and the adjusting of the seat ofthe passenger base on a determination that it is not necessary to pickup the fallen object.

In one embodiment of the present disclosure, based on the determinationthat it is not necessary to pick up the fallen object, the controllerremoves the object which is not necessary to pick up using a tool in thevehicle.

One embodiment of the present disclosure further includes a displayconfigured to display a location of the fallen object.

According to another aspect of the present disclosure, a method fordetecting a fallen object in a vehicle includes generating at least oneimage of an inside of the vehicle; identifying at least one passengerand at least one object from the image, and determining a location ofthe fallen object in response to a falling of the object in the vehicle;and providing the location of the fallen object via at least onecomponent in the vehicle and adjusting a seat of the passenger based onthe location of the fallen object and a condition of the passenger.

In one embodiment of the present disclosure, the adjusting of a seat ofthe passenger includes: setting a range where the passenger can pick upthe object based on the physical condition of the passenger including atleast one of arm length, leg length, or hand length of the passenger andin response to the location of the object being out of the set range,adjusting at least one of position, height, or angle of the seat of thepassenger.

In one embodiment of the present disclosure, the method may furtherinclude tracking the movement of the object from the periodicallygenerated image and determining, as the tracking result, that the objecthas fallen based on the object dropping by a predetermined distance ormore or dropping to a floor of the vehicle.

In one embodiment of the present disclosure, the method may furtherinclude tracking the movement of the object from the image generatedperiodically by the camera, and in response to the tracked object beingpredicted to fall based on at least one of movement or a shape of thetracked object, suggesting to move the object to a safe area, where itis determined that the tracked object will not fall, before the trackedobject falls.

In one embodiment of the present disclosure, the method may furtherinclude: suggesting to pick up the fallen object based on the vehiclestopping or being in a sector where the vehicle drives at apredetermined speed or lower; and suggesting to immediately pick up thefallen object based on the vehicle being expected to make a sharp turnor drive on an irregular road within a predetermined time or based onthe vehicle being expected to rapidly decelerate or accelerate, based onthe navigation information of the vehicle.

In one embodiment of the present disclosure, the method may include:based on the fallen object being a communication device capable of beingconnected to the component in the vehicle via short range wirelesscommunication; determining whether the object is connected to thecomponent in the vehicle and in response to the object being connectedto the component, as a determination result, temporarily stoppingproviding services in the fallen object and the component; and inresponse to the object being picked up by the passenger, controlling theobject and the component to continuously provide the service.

In one embodiment of the present disclosure, the method may include:identifying whether the passenger picks up the fallen object from theimage periodically generated; and restricting the adjusting of the seatand suggesting to pick up the fallen object in response to a command foradjusting the seat to a range where the fallen object becomes damaged isinput in a state where it is identified that the fallen object is notpicked up by the passenger.

In one embodiment of the present disclosure, the method may includedetermining, from the image of the fallen object, whether it isnecessary to pick up the fallen object by applying a deep neural networkmodel trained for determining whether the object is necessary for thepassenger, and restricting the providing of the location of the objectand the adjusting of the seat of the passenger based on a determinationthat it is not necessary to pick up the fallen object.

In addition, other methods and other systems for implementing thepresent disclosure, and a computer-readable medium for storing acomputer program for executing the above method may be further provided.

Other aspects, features, and advantages other than those described abovewill become apparent from the following drawings, claims, and thedetailed description of the present disclosure.

According to the present disclosure, when it is determined that anobject falls in the vehicle, a location of the fallen object is providedthrough a component in a vehicle or an internal display, and a seat ofthe passenger in the vehicle is adjusted based on the location of thefallen object and a physical condition (for example, arm length, leglength, or hand length of the passenger) of the passenger to easily pickup the fallen object.

According to the present disclosure, when the object is predicted tofall in the vehicle, movement of the object to a safe area which isdetermined that the object will not fall is suggested before the objectfalls to prevent the falling of the object in advance, therebypreventing damage to the object.

According to the present disclosure, a time to pick up the fallen objectmay be provided as guidance by a notification to pick up the fallenobject in the vehicle in a stable driving environment or a notificationto pick up the fallen object in advance when an unstable drivingenvironment is expected.

Further, according to the present disclosure, when the passenger doesnot pick up the fallen object even after completing the driving of thevehicle, notification information about the fallen object is providedthrough a component in a vehicle to prevent the fallen object from beingleft in the vehicle.

The effects of the present disclosure are not limited to those mentionedabove, and other effects not mentioned can be clearly understood bythose skilled in the art from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of the presentdisclosure will become apparent from the detailed description of thefollowing aspects in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a diagram illustrating a configuration of a fallen objectdetecting apparatus according to an embodiment of the presentdisclosure;

FIGS. 2 and 3 are diagrams illustrating an example of adjusting a seatof a passenger in a fallen object detecting apparatus according to anembodiment of the present disclosure;

FIG. 4 is a diagram illustrating an example of a tool in a vehiclemoving a fallen object in a fallen object detecting apparatus accordingto an embodiment of the present disclosure;

FIG. 5 is a diagram for explaining an example of guidance for picking upa fallen object by predicting a movement of a fallen object in a vehiclein a fallen object detecting apparatus according to an embodiment of thepresent disclosure;

FIG. 6 is diagram for explaining an example of guidance of a pick uptime of a fallen object in a fallen object detecting apparatus accordingto an embodiment of the present disclosure;

FIG. 7 is diagram for explaining an example of guidance for safelystoring an object predicted to fall in a vehicle in a fallen objectdetecting apparatus according to an embodiment of the presentdisclosure;

FIG. 8 is diagram for explaining an example of controlling a serviceprovided by a fallen object in a vehicle in a fallen object detectingapparatus according to an embodiment of the present disclosure; and

FIG. 9 is a flowchart illustrating a fallen object detecting methodaccording to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Advantages and features of the present disclosure and methods forachieving them will become apparent from the descriptions of aspectshereinbelow with reference to the accompanying drawings. However, thedescription of particular example embodiments is not intended to limitthe present disclosure to the particular example embodiments disclosedherein, but on the contrary, it should be understood that the presentdisclosure is to cover all modifications, equivalents and alternativesfalling within the spirit and scope of the present disclosure. Theexample embodiments disclosed below are provided so that the presentdisclosure will be thorough and complete, and also to provide a morecomplete understanding of the scope of the present disclosure to thoseof ordinary skill in the art. In the interest of clarity, not alldetails of the relevant art are described in detail in the presentspecification in so much as such details are not necessary to obtain acomplete understanding of the present disclosure.

The terminology used herein is used for the purpose of describingparticular example embodiments only and is not intended to be limiting.As used herein, the singular forms “a,” “an,” and “the” may be intendedto include the plural forms as well, unless the context clearlyindicates otherwise. The terms “comprises,” “comprising,” “includes,”“including,” “containing,” “has,” “having” or other variations thereofare inclusive and therefore specify the presence of stated features,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof. Furthermore, these terms such as “first,” “second,” and othernumerical terms, are used only to distinguish one element from anotherelement. These terms are generally only used to distinguish one elementfrom another.

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings. Like referencenumerals designate like elements throughout the specification, andoverlapping descriptions of the elements will not be provided.

The vehicle described in the present specification may include, but isnot limited to, a vehicle having an internal combustion engine as apower source, a hybrid vehicle having an engine and an electric motor asa power source, and an electric vehicle having an electric motor as apower source.

Further, the vehicle described in the present specification may be anautonomous vehicle.

Autonomous driving refers to a technology in which driving is performedautonomously, and an autonomous vehicle refers to a vehicle capable ofdriving without manipulation of a user or with minimal manipulation of auser. For example, autonomous driving may include a technology in whicha driving lane is maintained, a technology such as adaptive cruisecontrol in which a speed is automatically adjusted, a technology inwhich a vehicle automatically drives along a defined route, and atechnology in which a route is automatically set when a destination isset.

Specifically, the autonomous vehicle may include an autonomous drivingcontrol module to control an autonomous driving function and theautonomous driving control module may refer to a software module or achip obtained by implementing the software module as hardware. Theautonomous driving control module may be included inside as a componentof the autonomous vehicle, but may be configured as a separate hardwareat the outside of the autonomous to be connected thereto.

The autonomous vehicle may obtain status information of the autonomousvehicle, detect (recognize) surrounding environment and objects,generate map data, determine a travel path and navigation plan, ordetermine an operation, using sensor information obtained from varioustypes of sensors.

Here, the autonomous vehicle may use sensor information obtained from atleast one of sensors including lidar, radar, and camera to determine atravel path and navigation plan.

Specifically, the autonomous vehicle may recognize environments orobjects in an area where a field of view is blocked or an area separatedby a predetermined distance by receiving sensor information fromexternal devices or receive directly recognized information from theexternal devices.

The autonomous vehicle may perform the operations above by using alearning model built on at least one or more artificial neural networks.For example, the autonomous vehicle may recognize the surroundings andobjects by using the learning model and determine a travel line usingthe recognized surrounding information or object information. Here, thelearning model may be directly trained in the autonomous vehicle ortrained by an external device such as the AI server.

At this time, the autonomous vehicle may perform the operation bygenerating a result by employing the learning model directly but alsoperform the operation by transmitting sensor information to an externaldevice such as the AI server and receiving a result generatedaccordingly.

The autonomous vehicle may determine a travel path and navigation planusing at least one or more of map data, object information detected fromthe sensor information, or object information obtained from the externaldevice and controls a driving unit to navigate the autonomous vehicleaccording to the determined travel path and navigation plan.

The map data may include object identification information about variousobjects disposed in a space in which the autonomous vehicle navigates.For example, the map data may include object identification informationabout static objects such as streetlights, rocks and buildings andmovable objects such as vehicles and pedestrians. In addition, theobject identification information may include the name, type, distance,location, and so on.

Also, the autonomous vehicle may control the driving unit based on thecontrol/interaction of the user to perform the operation or navigate thespace. At this time, the autonomous vehicle may obtain intentioninformation of the interaction due to the user's motion or voice commandand perform an operation by determining a response based on the obtainedintention information.

FIG. 1 is a diagram illustrating a configuration of a fallen objectdetecting apparatus according to an embodiment of the presentdisclosure.

Referring to FIG. 1, a fallen object detecting apparatus 100 accordingto one embodiment of the present disclosure is a device which detectsthe falling of an object in the vehicle and is installed in the vehicle.The fallen object detecting apparatus 100 may include an camera 101, animage identifier 102, a controller 103, a display 104, and acommunicator 105.

The camera 101 may be, for example, one or more cameras, and maygenerate at least one image inside the vehicle. Such camera 101 maycapture the inside of the vehicle where a passenger (for example, adriver or occupant) and an object are present at a predetermined timeinterval (for example, every one second) to generate an image (forexample, a two-dimensional or three-dimensional image). The camera 101may be an existing camera installed in the vehicle but is not limitedthereto and may be a separate camera for monitoring the detection offallen objects.

The image identifier 102 may use a deep neural network trained torecognize humans and objects to extract the passenger and object fromthe image generated by the camera 101, and may identify at least onepassenger and at least one object from the image. When the object isextracted, the image identifier 102 may also extract the object type(for example, litter, a wallet, a smartphone, or wireless earphones).When a device capable of performing short range wireless communication(for example, Bluetooth or Wi-Fi) with a component in the vehicle isextracted as the object, the image identifier 102 may identify theobject as a communication device. Here, the device capable of performingshort range wireless communication may be any one of a smartphone,wireless earphones, a tablet PC, and a notebook computer.

When the passenger and the object are identified from the imagegenerated periodically by the camera 101, the image identifier 102 maytrack a motion (for example, a change of a coordinate of an upper body,a direction of a face, or arms) of the passenger and a movement of theobject from the image. The image identifier 102 may detect the movingdistance and direction of arms, legs, a head, and a face of thepassenger from the image by using a skeletonization algorithm to trackthe motion of the passenger. Further, the image identifier 102 may trackthe movement of the object and identify the location of the object bydetecting the moving distance and direction of the object from the imageusing an object detection algorithm.

As a tracking result, based on the object dropping by a predetermineddistance or more or dropping to a floor of the vehicle, the imageidentifier 102 may determine that the object has fallen. That is, basedon the object dropping by a predetermined distance or more or droppingto a floor of the vehicle as the object is separated or moved from acradle, a seat, or hands of the passenger, the image identifier 102 maydetermine that the object has fallen.

When the passenger carries the object, the image identifier 102 maydetermine that the object has fallen based on a distance between thehands of the passenger and the object being a predetermined interval ormore and based on the object dropping a predetermined distance or more,as the tracking result. That is, the image identifier 102 may determinewhether the object has fallen by further identifying whether the objecthas separated from the hands of the passenger in addition to the fallingdistance of the object. Accordingly, even though the object drops apredetermined distance or more in the vehicle, when the distance betweenthe hands of the passenger and the object is not a predeterminedinterval or more, the image identifier 102 may determine that the objecthas not fallen. As a result, even though the object drops apredetermined distance or more, based on the object being in contactwith the hands of the passenger, the image identifier 102 may determinethat the object has not fallen by recognizing that the passenger hasmoved his or her hand in a downward direction in a state of holding theobject in his or her hand.

When it is determined that the object has fallen in the vehicle based onan image periodically generated by the camera 101, the image identifier102 may identify a location of the fallen object based on the image.

The image identifier 102 may identify whether the passenger picks up thefallen object from the image periodically generated by the camera 101.Specifically, as a result of tracking the passenger and the object inthe image periodically generated by the camera 101, based on a distancebetween the hand of the passenger and the fallen object being less thanthe predetermined interval (for example, 0.1 mm), or the hand of thepassenger being in contact with the object, and the fallen object risinga predetermined distance or more, the image identifier 102 may determinethat the passenger has picked up the fallen object.

Such image identifier 102 may identify at least one passenger and atleast one object from the image, and in response to the falling of theobject in the vehicle, may determine the position of the fallen object.The determined position of the fallen object may be provided to thecontroller 103.

The controller 103 may provide falling information about the object andthe location of the fallen object determined by the image identifier 102via at least one component in the vehicle (for example, a speaker or anaudio video navigation (AVN) screen) or the display 104. For example, inresponse to a cellular phone falling in the vehicle, the controller 103may provide a message about the fallen object informing that thecellular phone has fallen to the right side of the gearbox console via aspeaker in the vehicle or provide an image displaying the location ofthe cellular phone to the right side of the gearbox console via thedisplay 104.

Thereafter, the controller 103 may adjust the seat of the passenger inthe vehicle based on the location of the object and the condition (forexample, physical condition) of the passenger. The controller 103 mayadjust the seat of the passenger to a distance such that the hand orfoot of the passenger is able to reach the location of the fallenobject. In addition, the controller 103 may set a range (for example, alocation accessible by the hand or foot of the passenger) where thepassenger is able to pick up the object based on the physical conditionof the passenger including at least one of arm length, leg length, orhand length of the passenger, and in response to the location of theobject being out of the set range (that is, the hand or foot of thepassenger not being able to reach the fallen object), may adjust atleast one of position, height, or angle of the seat of the passenger toallow the passenger to take a position to easily pick up the fallenobject.

Before adjusting the seat of the passenger, when it is identified thatthe passenger searches for the object via the image identifier 102, thecontroller 103 may move the seat when the passenger intends to pick upthe object by adjusting the seat of the passenger in the vehicle. Inthis case, the image identifier 102 may determine whether the passengerlowers his or her head based on a motion tracking result of thepassenger and based on a determination that the passenger has loweredhis or her head, may confirm that the passenger searches for the object.Here, the image identifier 102 may determine whether the passenger haslowered his or her head based on how much the face (or the head) of thepassenger as moved and a direction of movement. As another example, theimage identifier 102 may identify whether the passenger searches for theobject from the image by using a deep neural network model trained todetermine whether the passenger searches for the object to pick up thefallen object.

When the seat of the passenger is adjusted, the controller 103 mayrestrict an adjusting range of the seat of the passenger to within arange where the object does not become damaged, based on the location ofthe fallen object to prevent damage to the object caused by the movementof the adjusted seat. That is, the controller 103 may prevent the seatfrom moving to a location where the object may be damaged and allow thepassenger to pick up an object that is intact.

Further, in response to the location of the fallen object being out ofthe range where the passenger may pick up the object, the controller 103may move the fallen object to a range where the fallen object is able tobe picked up by using a tool (for example, a wireless cleaner) installedin the vehicle. As another example, in response to the location of thefallen object being out of the range where the passenger is able to pickup the object, the controller 103 may change a driving direction or aspeed of the vehicle to move the fallen object to a pick up range orprovide guidance for changing the driving direction or the speed via acomponent in the vehicle (for example, internal display).

For example, when the fallen object is located on the left side of thepick up range, the controller 103 may move the fallen object to theright by using the tool or by making a left turn (or a left lane change)to move the fallen object to the pick up range.

The controller 103 may control the tool or the driving direction (speed)the vehicle before adjusting the seat, but the present disclosure is notlimited thereto. When the fallen object remains out of the range thatthe passenger is able to pick up the object even after primarilyadjusting the seat, the controller 103 may secondarily control the toolor the driving direction (speed) of the vehicle to allow the passengerto easily pick up the fallen object. Here, the range that the passengeris able to pick up the object may vary before and after adjusting theseat of the passenger.

The controller 103 may predict whether the tracked object falls based onat least one of movement or a shape of the tracked object from the imageperiodically generated. In this case, in response to the tracked objectbeing predicted to fall, the controller 103 may suggest to move thetracked object to a safe area where it is determined that the trackedobject will not fall, before the tracked object falls. For example, whenthe object moves (sways in both directions) more than a set value or ashape of the object is circular (or cylindrical), the controller 103 maypredict that the object will fall. In this case, before the objectfalls, the controller 103 may determine a storage container in thevehicle and a cup holder to be a safe area where it is determined thatthe object will not fall, and may suggest moving the object to thedetermined safe area. In this case, the controller 103 may suggestmoving the object to the safe area via the component in the vehicle (forexample, a speaker or an AVN screen) or the display 104.

Furthermore, after the object falls in the vehicle, when the object ismoved to a same area where the object was located prior to falling, thecontroller 103 may predict that the object will fall again and suggestmoving the object to the safe area.

The controller 103 may provide guidance of a time to pick up the fallenobject by notifying the passenger to pick up the fallen object in thevehicle in a stable driving environment or notifying the passenger topick up the fallen object in advance when an unstable drivingenvironment is expected. Specifically, when the vehicle stops or is in asector in which the vehicle drives at a predetermined speed or lower,the controller 103 may suggest picking up the fallen object.Furthermore, when the vehicle is expected to make a sharp turn or driveon an irregular road within a predetermined time, or when the vehicle isexpected to rapidly decelerate or accelerate, based on the navigationinformation of the vehicle, the controller 103 may suggest immediatelypicking up the fallen object. Here, the controller 103 may obtainnavigation information of the vehicle (for example, a road on which thevehicle drives, a road shape, a road surface, and a speed of the vehicleon the road) corresponding to the destination from the current locationof the vehicle from the map server via the communicator 105.

In a state where the image identifier 102 identifies that the fallenobject has not been picked up by the passenger, in response to a commandfor adjusting the seat to a range where the fallen object is capable ofbeing damaged is input, the controller 103 may restrict the adjusting ofthe seat and suggests picking up the fallen object. That is, when thecommand for adjusting the seat is input, the controller 103 maydetermine that the movement range of the seat adjusted in response tothe adjusting command includes the location of the fallen object. Whenit is determined that the location of the fallen object is included inthe movement range of the seat, the controller 103 may restrict theadjusting of the seat and recommend picking up the fallen object due tothe determination that the adjusting of the seat may damage the fallenobject.

Further, the controller 103 may determine whether it is necessary topick up the fallen object via the image identifier 102. Here, the imageidentifier 102 may determine whether it is necessary to pick up thefallen object depending on whether the type of the fallen object isincluded in a set list or determine whether it is necessary to pick upthe fallen object by applying a deep neural network model trained todetermine, from the image of the fallen object, whether the fallenobject is necessary for the passenger.

As a determination result, based on a determination that it is notnecessary to pick up the fallen object (for example, litter), thecontroller 103 may restrict (omit) the providing of the location of theobject and adjust the seat of the passenger. Further, as thedetermination result, based on the determination that it is notnecessary to pick up the fallen object, the controller 103 may removethe object by using a tool in the vehicle (for example, a wirelesscleaner).

Based on the fallen object being a communication device capable of beingconnected with the component in the vehicle via short range wirelesscommunication, the controller 103 may identify whether the fallen objectis connected to the component in the vehicle via short range wirelesscommunication. Specifically, the controller 103 may requestsidentification information of the fallen object via the communicator105. When the object identification information received from the objectcoincides with identification information of a device connected with thecomponent in the vehicle as a response for the request, the controller103 may identify that the object is connected with the component viashort range wireless communication. In this case, the controller 103 mayreceive identification information of a device connected with thecomponent, from the component in the vehicle, via the communicator 105.

As an identifying result, based on the fallen object being connectedwith the component in the vehicle via short range wirelesscommunication, the controller 103 may temporarily stop providingservices (for example, video playback or game play) of the fallen objectand the component, and in response to the image identifier 102determining that the object has been picked up by the passenger, thecontroller 103 may control the object and the component to continuouslyprovide services. That is, when an object providing a service hasfallen, the service is not normally provided to the passenger, thereforethe controller 103 may temporarily stop the service provided by theobject and the component, and when the passenger picks up the fallenobject to continue the providing of the normal service, the controller103 may resume the providing of the service of the object and thecomponent.

Further, if the passenger does not pick up the fallen object, when thedriving of the vehicle is completed, the controller 103 providesnotification information about the fallen object (for example, theobject falls on the floor) through the component in the vehicle or thedisplay 104 so that even though the passenger does not pick up theobject immediately falling the object, the presence of the fallen objectis notified. Therefore, the passenger may pick up the object withoutforgetting.

The display 104 may display information about the object (for example,the location of the fallen object) or provide guidance for the object(for example, guidance for moving the object to the safe area orguidance for picking up the fallen object). The display 104 may be adisplay for providing the information about the object, but is notlimited thereto and may be replaced with an existing component in thevehicle (for example, an AVN screen).

The communicator 105 may receive navigation information of the vehiclefrom a map server, receive identification information from the fallenobject in the vehicle, or receive identification information of aconnected device from the component in the vehicle.

FIGS. 2 and 3 are diagrams illustrating an example of adjusting a seatof a passenger in a fallen object detecting apparatus according to anembodiment of the present disclosure.

Referring to FIG. 2, when it is determined that an object has fallen inthe vehicle from an image obtained by periodically capturing the insideof the vehicle, the fallen object detecting apparatus may provide thelocation of the fallen object via the component (for example, a speakeror an AVN screen) or the display. In this case, the fallen objectdetecting apparatus may adjust at least one of position (for example,front or rear), height (for example, up or down), or angle (for example,a direction) of the seat of the passenger in the vehicle, based on thelocation of the fallen object and a physical condition (for example, armlength, leg length, or hand length of the passenger) of the passenger.

For example, when it is determined that “wireless earphones” 201 havefallen in the vehicle, the fallen object detecting apparatus providesthe location of the fallen “wireless earphones” 201 via a speaker in thevehicle and adjusts at least one of a front/rear position 202, up/downheight 203, or left/right angle 204 of the seat of the passenger so thatthe hand of the passenger is able to reach the fallen “wirelessearphones” 201. Therefore, the passenger may easily pick up the“wireless earphones” 201.

As illustrated in FIG. 3, the fallen object detecting apparatus mayrestrict the adjusting range of the seat of the passenger to within arange where the “wireless earphones” 201 will not become damaged, basedon the location of the falling “wireless earphones” 201, so that theadjusting range of the seat does not intrude the range where the“wireless earphones” 201 will become damaged. Therefore, damage to the“wireless earphones” 201 due to the adjusting of the seat may beprevented.

FIG. 4 is a diagram illustrating an example of a tool in a vehiclemoving a fallen object in a fallen object detecting apparatus accordingto an embodiment of the present disclosure.

Referring to FIG. 4, when the location of the fallen object in thevehicle is out of the range where the passenger is able to pick up theobject (for example, a location accessible by the hand or foot of thepassenger), the fallen object detecting apparatus may move the fallenobject to a range where the fallen object is able to be picked up byusing a tool installed in the vehicle.

For example, when the location of “wireless earphones” 401 which hasfallen in the vehicle is out of a range 402 where the passenger is ableto pick up the object, the fallen object detecting apparatus may controla “wireless cleaner” 403 installed in the vehicle to push and move the“wireless earphones” 401 in the range where the passenger is able topick up the object. Thus, the passenger may easily pick up the “wirelessearphones” 401.

FIG. 5 is a diagram illustrating an example of guidance for picking up afallen object by predicting a movement of a fallen object in a vehiclein a fallen object detecting apparatus according to an embodiment of thepresent disclosure.

Referring to FIG. 5, the fallen object detecting apparatus may identifythe movement of the fallen object based on a result of tracking anobject in an image which is periodically generated by the camera, in avehicle being driving. In this case, the object may move differentlyaccording to the weight and shape of the object.

For example, when the objects which have fallen in the vehicle beingdriven are a “briefcase” 501, “wireless earphones” 502, and a “ball”503, the circular “ball” 503, which is light in weight and able to berolled, may move the farthest, the “wireless earphones” 502, which islighter in weight than the “briefcase” 501, may move farther than the“briefcase” 501 but not as far as the “wireless earphones” 502, and the“briefcase” 501, which is heavier than the other objects, may move theleast.

Therefore, the fallen object detecting apparatus may provide guidancefor picking up the objects based on weight and shape in descending orderof the moving distance of the objects, that is, the “ball” 503, the“wireless earphone” 502, and the “briefcase” 501, via the component inthe vehicle, or provide guidance for immediately picking up the “ball”503, which may move the farthest, via the component in the vehicle.

FIG. 6 is diagram for explaining an example of guidance of a pick uptime of a fallen object in a fallen object detecting apparatus accordingto an embodiment of the present disclosure.

Referring to FIG. 6, the fallen object detecting apparatus may provideguidance of a pick up time for the object which has fallen in a vehicle.For example, when the vehicle is expected to make a sharp turn or driveon an irregular road within 20 seconds or when the vehicle is expectedto rapidly decelerate or accelerate, based on the navigation informationof the vehicle, the fallen object detecting apparatus may suggestimmediately picking up the object which has fallen in the vehicle.

Further, when a vehicle 601 stops or is in a sector where the vehicledrives at a predetermined speed or lower, the fallen object detectingapparatus may suggest picking up the fallen object.

As another example, as the vehicle drives on a winding road 602, whenthe object falls at a time 603 when the vehicle enters the winding road602, the fallen object detecting apparatus may suggest picking up thefallen object at a time 604 when the vehicle exits the winding road 602.

FIG. 7 is diagram for explaining an example of guidance for safelystoring an object predicted to fall in a vehicle in a fallen objectdetecting apparatus according to an embodiment of the presentdisclosure.

Referring to FIG. 7, when the object is predicted to fall inconsideration of at least one of the movement of the object in thevehicle or the shape of the object, the fallen object detectingapparatus may suggest keeping the object in a safe location before theobject falls.

For example, the fallen object detecting apparatus may periodicallycapture the inside of the vehicle to generate an image, and when a“cellular phone” 701 is extracted from the image, may track the“cellular phone” 701 to identify the movement of the “cellular phone”701. In this case, when the “cellular phone” 701 moves more than a setvalue (for example, 10 cm) or the “cellular phone” 701 is located on anedge (an area within a predetermined interval from an edge of thesurface of a dashboard) of the surface of a dashboard 702, the fallenobject detecting apparatus may predict that the “cellular phone” 701will fall. Therefore, the fallen object detecting apparatus may suggestmoving the object to a safe area where it is determined that the“cellular phone” 701 will not fall, for example, a console box 703between the seats, to prevent the object from falling in advance.Therefore, the object does not become damaged. In this case, the fallenobject detecting apparatus may output a message to suggest moving theobject such as, “The cellular phone may fall. Please move the cellularphone to a safe area such as the console box,” via the speaker in thevehicle.

FIG. 8 is diagram for explaining an example of controlling a serviceprovided by a fallen object in a vehicle in a fallen object detectingapparatus according to an embodiment of the present disclosure.

Referring to FIG. 8, the fallen object detecting apparatus may identifywhether the fallen object in the vehicle is connected to the componentin the vehicle via short range wireless communication. For example, when“wireless earphones” 801 fall in the vehicle, the fallen objectdetecting apparatus may request identification information to thefalling “wireless earphones” 801. When the identification informationreceived from the “wireless earphones” 801 as a response to the requestcoincides with identification information of the device connected to thecomponent in the vehicle, for example, a display 802 (or AVN screen),the fallen object detecting apparatus may confirm that the “wirelessearphones” 801 and the display 801 in the vehicle are connected viashort range wireless communication.

When the fallen object is connected to the display 802 in the vehiclevia short range wireless communication, as the identifying result, thefallen object detecting apparatus may temporarily stop the service (forexample, video playback) provided by the fallen “wireless earphones” 801and the display 802 in the vehicle. When the “wireless earphones” 801 ispicked up by the passenger, the fallen object detecting apparatus maycontrol the “wireless earphones” 801 and the display 802 in the vehicleto continuously provide the service. That is, when the “wirelessearphones” 801 which provides a service falls, the service may no longerbe normally provided to the passenger, therefore the fallen objectdetecting apparatus may temporarily stop the service (for example, videoplayback) provided by the falling “wireless earphones” 801 and thedisplay 802 in the vehicle, and as the passenger picks up the fallen“wireless earphones” 801, the “wireless earphones” 801 and the display802 in the vehicle may resume the providing of the service.

FIG. 9 is a flowchart illustrating a fallen object detecting methodaccording to an embodiment of the present disclosure.

Referring to FIG. 9, in step S901, a fallen object detecting apparatus,for example, may use at least one camera to generate at least one imageof the inside of a vehicle. The fallen object detecting apparatus mayperiodically capture the inside of the vehicle where a passenger (forexample, a driver or an occupant) and an object are present to generatean image. Here, the fallen object detecting apparatus may capture theinside of the vehicle where a passenger (for example, a driver or anoccupant) and an object are present at a predetermined time interval(for example, every one second) by using an internal camera or anexisting camera installed in the vehicle to generate an image (forexample, a two-dimensional or three-dimensional image).

The fallen object detecting apparatus may use a deep neural networktrained to recognize humans and objects to extract the passenger andobject from the image generated by the camera 101, and may identify atleast one passenger and at least one object from the image. In thiscase, when the object is extracted, the fallen object detectingapparatus may also extract an object type (for example, litter, awallet, a smartphone, or wireless earphones). Here, when any one deviceof a smartphone, wireless earphones, a tablet PC, and a notebookcomputer, which are capable of performing short range wirelesscommunication with the component in the vehicle, is extracted as theobject, the fallen object detecting apparatus may confirm the object asa communication device.

When the passenger and the object are identified from the image, thefallen object detecting apparatus may track a motion (for example, achange of a coordinate of an upper body, a direction of a face, andarms) of the passenger and the movement of the object from the imagegenerated periodically. In this case, the fallen object detectingapparatus may detect the moving distance and direction of arms, legs, ahead, and a face of the passenger from the image by using askeletonization algorithm to track the motion of the passenger. Further,the fallen object detecting apparatus may track the movement of theobject and identify the location of the object by detecting the movingdistance and direction of the object from the image using an objectdetection algorithm.

The fallen object detecting apparatus may predict whether the trackedobject will fall based on at least one of movement or a shape of thetracked object. In this case, in response to the tracked object beingpredicted to fall, the fallen object detecting apparatus may suggestmoving the tracked object to a safe area where it is determined that thetracked object will not fall via the component in the vehicle (orinternal display) to prevent the object from falling. For example, whenthe object moves (shakes) more than a set value or a shape of the objectis circular (or cylindrical), the fallen object detecting apparatus maypredict that the object will fall. In this case, the fallen objectdetecting apparatus may determine that a storage container in thevehicle and a cup holder as a safe area, and may suggest moving theobject to the determined safe area to keep the object intact. In thiscase, the fallen object detecting apparatus may suggest moving theobject to the safe area via the component in the vehicle (for example, aspeaker or an AVN screen) or the internal display.

As the tracking result, based on the object dropping by a predetermineddistance or more or dropping to the floor of the vehicle, the fallenobject detecting apparatus may determine that the object has fallen.That is, based on the object being separated or moved from a cradle, aseat, or the hands of the passenger by a predetermined distance or moreor dropped to the floor, the fallen object detecting apparatus maydetermine that the object has fallen.

When the passenger carries the object, the fallen object detectingapparatus may determine that the object has fallen based on a distancebetween the hands of the passenger and the object being a predeterminedinterval or more and based on the object dropping a predetermineddistance or more, as the tracking result. That is, the fallen objectdetecting apparatus may determine whether the object has fallen byfurther identifying whether the object has separated from the hands ofthe passenger in addition to the falling distance of the object.

In step S902, in response to a determination that the object has fallenin the vehicle, the fallen object detecting apparatus may determine thelocation of the fallen object. In this case, the fallen object detectingapparatus may provide the determined position of the fallen object viaat least one component in the vehicle. In this case, the fallen objectdetecting apparatus may set a range where the passenger is able to pickup the object (for example, an accessible location by the hand or footof the passenger) based on the physical condition of the passengerincluding at least one of arm length, leg length, or hand length of thepassenger, and may identify whether when the location of the fallenobject is out of the set range.

In step S903, in response to the location of the fallen object being outof the range where the passenger is able to pick up the fallen object,the fallen object detecting apparatus may adjust the seat of thepassenger in the vehicle in step S904. In this case, the fallen objectdetecting apparatus may adjust the seat of the passenger based on thelocation of the fallen object and the condition of the passenger.Specifically, the fallen object detecting apparatus may adjust at leastone of position, height, or angle of the seat of the passenger in thevehicle to a distance where the hand or foot of the passenger reachesthe location of the fallen object. When the seat of the passenger isadjusted, the fallen object detecting apparatus may restrict anadjusting range of the seat of the passenger within a range where theobject does not become damaged, based on the location of the fallenobject to prevent the damage of the object due to the adjusted seat.

Before adjusting the seat of the passenger, the fallen object detectingapparatus may identify whether the passenger searches for the object,and when it is confirmed that the passenger searches for the object, thefallen object detecting apparatus may move the seat by adjusting theseat of the passenger in the vehicle when the passenger intends to pickup the object. In this case, the fallen object detecting apparatus maydetermine whether the passenger lowers his or her head based on a motiontracking result of the passenger in the image generated by the camera,and when it is determined that the passenger lowers his or her head, mayconfirm that the passenger searches for the object.

Further, in response to the location of the fallen object being out ofthe range where the passenger may pick up the object, the fallen objectdetecting apparatus may move the fallen object by using a tool (forexample, a wireless cleaner) installed in the vehicle to a range wherethe fallen object is able to be picked up. As another example, thefallen object detecting apparatus may change the driving direction ofthe vehicle or the speed of the vehicle so that the fallen object movesto the pick up range or provide guidance for changing the drivingdirection of the vehicle or the speed of the vehicle via the componentin the vehicle (or an internal display). For example, when the locationof the fallen object is the right side of the pick up range, the fallenobject detecting apparatus may move the fallen object to the left byusing the tool or making a right turn (or right lane change) to move thefallen object to the pick up range.

When range where the passenger is able to pick up the fallen objectincludes the location of the fallen object, in step S903, the fallenobject detecting apparatus may omit the step S904.

Thereafter, the fallen object detecting apparatus may identify whetherthe passenger picks up the fallen object. Specifically, when the hand ofthe passenger is in contact with the fallen object and the fallen objectrises by a set distance or more as a tracking result of the passengerand the object in the image which is periodically generated by thecamera, the fallen object detecting apparatus may determine that thepassenger picks up the fallen object.

In step S905, when it is identified that the passenger has not picked upthe fallen object, the fallen object detecting apparatus may provideguidance of a time to pick up the fallen object in step S906. That is,the fallen object detecting apparatus may provide guidance of a time topick up the fallen object by notifying to pick up the object falling inthe vehicle in a stable driving environment or notifying to pick up thefallen object in advance in the vehicle when an unstable drivingenvironment is expected. Specifically, when the vehicle stops or is in asector in which the vehicle drives at a predetermined speed or lower,the fallen object detecting apparatus may suggest to pick up the fallenobject. Further, based on the vehicle expected to make a sharp turn ordrive on an irregular road within a predetermined time or when thevehicle is expected to rapidly decelerate or accelerate, based on thenavigation information of the vehicle, the fallen object detectingapparatus may suggest to immediately pick up the fallen object.

In a state where the fallen object detecting apparatus identifies thatthe fallen object has not been picked up by the passenger, in responseto a command for adjusting the seat to a range where the fallen objectmay become damaged is input, the fallen object detecting apparatus mayrestrict the adjusting of the seat and suggest picking up the fallenobject. That is, when the command for adjusting the seat is input, thefallen object detecting apparatus may determine that the movement rangeof the seat adjusted in response to the adjusting command includes thelocation of the fallen object. When it is determined that the movementrange of the seat includes the location of the fallen object, the fallenobject detecting apparatus may determine that the adjusting of the seatmay cause damage to the fallen object, and restrict the adjusting forthe seat and recommend to pick up the fallen object through thecomponent in the vehicle (for example, an internal display).

Further, the fallen object detecting apparatus may determine whether itis necessary to pick up the fallen object, and based on a determinationthat it is not necessary to pick up the fallen object, may restrict theadjusting the seat of the passenger and provide the location of theobject. Here, the fallen object detecting apparatus may determinewhether it is necessary to pick up the fallen object depending onwhether the type of the fallen object is included in a set list ordetermine whether it is necessary to pick up the fallen object byapplying a deep neural network model trained to determine whether thefallen object is necessary for the passenger to the image of the fallenobject.

Further, as the determination result, in response to a determinationthat it is not necessary to pick up the fallen object, the fallen objectdetecting apparatus may remove the object which is not necessary to pickup by using a tool in the vehicle.

Based on the fallen object being a communication device which performsshort range wireless communication with the component in the vehicle,the fallen object detecting apparatus may identify whether the fallenobject is connected to the component in the vehicle through short rangewireless communication. Specifically, the fallen object detectingapparatus may request identification information of the fallen object.When the object identification information received from the objectcoincides with identification information of a device connected in thecomponent in the vehicle as the response for the request, the fallenobject detecting apparatus may confirm that the object is connected withthe component by the short range wireless communication.

As a identifying result, in response to the fallen object beingconnected to the component in the vehicle via short range wirelesscommunication, the fallen object detecting apparatus may temporarilystop the providing of services (for example, video playback or gameplay) in the fallen object and the component, and in response to theobject being picked up by the passenger, the fallen object detectingapparatus may control the object and the component to continuouslyprovide the service. That is, when an object which provides a servicefalls, as the passenger is not normally provided with the service, thefallen object detecting apparatus may temporarily stops the serviceprovided by the object and the component and when the passenger picks upthe fallen object so that the service is normally provided, the fallenobject detecting apparatus may resume the providing of the service inthe object and the component.

Further, when the driving of the vehicle is completed, the fallen objectdetecting apparatus may identify whether the passenger picks up thefallen object and based on the passenger not picking up the object, thefallen object detecting apparatus provides the notification informationabout the fallen object (for example, the objects falls on the floor)through the component in the vehicle, so that the fallen object is notleft in the vehicle.

The embodiments of the present disclosure described above may beimplemented through computer programs executable through variouscomponents on a computer, and such computer programs may be recorded incomputer-readable media. Examples of the computer readable medium mayinclude magnetic media such as a hard disk drives (HDD), floppy disksand a magnetic tape, optical media such as CD-ROMs and DVDs,magneto-optical media such as floptical disks, or hardware devices suchas ROMs, RAMs, and flash memories specifically configured to store andexecute program commands.

Meanwhile, the computer programs may be those specially designed andconstructed for the purposes of the present disclosure or they may be ofthe kind well known and available to those skilled in the computersoftware arts. Examples of program code include both a machine code,such as produced by a compiler, and a higher-level code that may beexecuted by the computer using an interpreter.

In the present application (especially, in the appended claims), the useof the terms “the,” “the above-mentioned,” and/or other terms similarthereto may correspond to singular meaning, plural meaning, or both ofthe singular meaning and the plural meaning as necessary. Also, itshould be understood that any numerical range recited herein is intendedto include all sub-ranges subsumed therein (unless expressly indicatedotherwise) and accordingly, the disclosed numeral ranges include everyindividual value between the minimum and maximum values of the numeralranges.

The above-mentioned steps constructing the method disclosed in thepresent disclosure may be performed in a proper order unless explicitlystated otherwise. However, the scope or spirit of the present disclosureis not limited thereto. All examples described herein or the termsindicative thereof (“for example,” “such as”) used herein are merely todescribe the present disclosure in greater detail. Therefore, it shouldbe understood that the scope of the present disclosure is not limited tothe example embodiments described above or by the use of such termsunless limited by the appended claims. In addition, technical ideas ofthe present disclosure can also be readily implemented by those skilledin the art according to various conditions and factors within the scopeof the appended claims to which various modifications, combinations, andchanges are added, or equivalents thereof.

The present disclosure is not limited to the example embodimentsdescribed above, and rather intended to include the following appendedclaims, and all modifications, equivalents, and alternatives fallingwithin the spirit and scope of the following claims.

What is claimed is:
 1. An apparatus for detecting a fallen object in avehicle, the apparatus comprising: a camera configured to generate atleast one image of an inside of the vehicle; an image identifierconfigured to identify at least one passenger and at least one objectfrom the image, and to determine a location of the fallen object inresponse to a falling of the object in the vehicle; and a controllerconfigured to provide the determined location of the fallen object viaat least one component located in the vehicle and adjust a seat of thepassenger based on the location of the fallen object and a condition ofthe passenger.
 2. The fallen object detecting apparatus according toclaim 1, wherein the controller is configured to set a range in whichthe passenger is capable of picking up the object based on the physicalcondition of the passenger including at least one of arm length, leglength, or hand length, and in response to the location of the objectbeing out of the set range, adjust at least one of position, height, orangle of the seat of the passenger.
 3. The fallen object detectingapparatus according to claim 1, wherein the image identifier isconfigured to track a movement of the object from the image generatedperiodically by the camera and determine, as a tracking result, that theobject has fallen based on the object dropping by a predetermineddistance or more or dropping to a floor of the vehicle.
 4. The fallenobject detecting apparatus according to claim 1, wherein the controlleris configured to restrict an adjusting range of the seat of thepassenger within a range in which the object does not become damaged,based on the location of the fallen object.
 5. The fallen objectdetecting apparatus according to claim 1, wherein the image identifieris configured to track the movement of the object from the imagegenerated periodically by the camera, and wherein in response to thetracked object being predicted to fall based on at least one of movementor shape of the tracked object, the controller is configured to suggestmoving the object to a safe area, where it is determined that thetracked object will not fall, before the tracked object falls.
 6. Thefallen object detecting apparatus according to claim 1, wherein thecontroller is configured to suggest picking up the fallen object basedon the vehicle stopping or being in a sector where the vehicle drives ata predetermined speed or lower, or the controller is configured tosuggest immediately picking up the fallen object based on the vehiclebeing expected to turn sharply or drive on an irregular road within apredetermined time or based on the vehicle being expected to rapidlydecelerate or accelerate, based on navigation information of thevehicle.
 7. The fallen object detecting apparatus according to claim 1,wherein based on the fallen object being a communication device capableof being connected to the component in the vehicle via short rangewireless communication, the controller is configured to determinewhether the object is connected to the component in the vehicle, and asa determination result, in response to the object being connected to thecomponent, temporarily stop providing services of the fallen object andthe component, and in response to the object being picked up by thepassenger, configured to control the object and the component tocontinuously provide the service.
 8. The fallen object detectingapparatus according to claim 1, wherein the image identifier isconfigured to identify whether the passenger picks up the fallen objectfrom the image periodically generated by the camera, and wherein thecontroller is configured to restrict an adjusting of the seat andsuggest picking up the fallen object in response to a command foradjusting the seat to a range in which the fallen object becomes damagedbeing input in a state where it is identified that the fallen object isnot picked up by the passenger.
 9. The fallen object detecting apparatusaccording to claim 1, wherein in response to the location of the fallenobject being out of the range in which the passenger is capable ofpicking up the object, the controller is configured to move the fallenobject to a pick up range by using a tool installed in the vehicle orchange a driving direction or speed of the vehicle to move the fallenobject to the pick up range.
 10. The fallen object detecting apparatusaccording to claim 1, wherein the image identifier is configured todetermine, from the image of the fallen object, whether it is necessaryto pick up the fallen object by applying a deep neural network modeltrained for determining whether the object is necessary for thepassenger, and the controller is configured to restrict the providing ofthe location of the object and the adjusting of the seat of thepassenger based on a determination that it is not necessary to pick upthe fallen object.
 11. The fallen object detecting apparatus accordingto claim 10, wherein based on the determination that it is not necessaryto pick up the fallen object, the controller is configured to remove theobject which is not necessary to pick up using a tool in the vehicle.12. The fallen object detecting apparatus according to claim 1, furthercomprising: a display configured to display a location of the fallenobject.
 13. A method for detecting a fallen object in a vehicle, themethod comprising: generating at least one image of an inside of thevehicle; identifying at least one passenger and at least one object fromthe image, and determining a location of the fallen object in responseto a falling of the object in the vehicle; and providing the location ofthe fallen object via at least one component in the vehicle andadjusting a seat of the passenger based on the location of the fallenobject and a condition of the passenger.
 14. The fallen object detectingmethod according to claim 13, wherein the adjusting of the seat of thepassenger comprises: setting a range in which the passenger is capableof picking up the object based on the physical condition of thepassenger including at least one of arm length, leg length, or handlength of the passenger, and in response to the location of the objectbeing out of the set range, adjusting at least one of position, height,or angle of the seat of the passenger.
 15. The fallen object detectingmethod according to claim 13, further comprising: tracking a movement ofthe object from the periodically generated image and determining, as atracking result, that the object has fallen based on the object droppingby a predetermined distance or more or dropping to a floor of thevehicle.
 16. The fallen object detecting method according to claim 13,further comprising: tracking the movement of the object from theperiodically generated image; and in response to the tracked objectbeing predicted to fall based on at least one of movement or shape ofthe tracked object, suggesting to move the tracked object to a safearea, where it is determined that the tracked object will not fall,before the tracked object falls.
 17. The fallen object detecting methodaccording to claim 13, further comprising: suggesting to pick up thefallen object based on the vehicle stopping or being in a sector wherethe vehicle drives at a predetermined speed or lower; or suggesting toimmediately pick up the fallen object based on the vehicle beingexpected to turn sharply or drive on an irregular road within apredetermined time or based on the vehicle being expected to rapidlydecelerate or accelerate, based on navigation information of thevehicle.
 18. The fallen object detecting method according to claim 13,further comprising: based on the fallen object being a communicationdevice capable of being connected to the component in the vehicle viashort range wireless communication, determining whether the object isconnected to the component in the vehicle, and as a determinationresult, in response to the object being connected to the component,temporarily stopping providing of services of the fallen object and thecomponent, and controlling the object and the component to continuouslyprovide the service in response to the object being picked up by thepassenger.
 19. The fallen object detecting method according to claim 13,further comprising: identifying whether the passenger picks up thefallen object from the image periodically generated; and restricting theadjusting of the seat and suggesting to pick up the fallen object inresponse to a command for adjusting the seat to a range in which thefallen object becomes damaged is input in a state where it is identifiedthat the fallen object is not picked up by the passenger.
 20. The fallenobject detecting method according to claim 13, further comprising:determining, from the image of the fallen object, whether it isnecessary to pick up the fallen object by applying a deep neural networkmodel trained for determining whether the object is necessary for thepassenger, and restricting the providing of the location of the objectand the adjusting of the seat of the passenger based on a determinationthat it is not necessary to pick up the fallen object.